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Abstract 

An alternative to extrinsic information transfer (EXIT) charts called mean squared error (MSE) charts that use 
a measure related to the MSE instead of mutual information is proposed. Using the relationship between mutual 
information and minimum mean squared error (MMSE), a relationship between the rate of any code and the area 

^ \ under a plot of MSE versus signal to noise ratio (SNR) is obtained, when the log likelihood ratio's (LLR) can 

Q '■ 

be assumed to be from a Gaussian channel. Using this result, a theoretical justification is provided for designing 
concatenated codes by matching the EXIT charts of the inner and outer decoders, when the LLRs are Gaussian 
which is typically assumed for code design using EXIT charts. Finally, for the special case of AWGN channel it 
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for erasure channels to the Gaussian channel. 
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^ . I. Introduction 

An Extrinsic Information Transfer (EXIT) chart is an insightful and extremely useful tool to analyze 
iterative decoding schemes. In an EXIT chart, the mutual information transfer characteristics of the 
component decoders is plotted to study the convergence behavior graphically. 

Consider a serial concatenation of convolutional codes shown in Fig. [lj For this case, EXIT charts 
have the following two properties. One, the EXIT curve of the inner code should lie above the EXIT 
curve (after reflecting about the line y = x) of the outer code for the iterations to converge to the correct 
codeword. Two, the area under the EXIT chart is related to the rate of the code. If the a priori information 
is assumed to be from an erasure channel, Ashikhmin et al [2] showed that for any code of rate R, the 
area under the exit curve is 1 — R. Based on these properties it is easy to see that an optimum code can 
be designed by matching the EXIT charts. Recently, this technique has been used to design codes that 



is shown that any capacity achieving code has an EXIT curve that is flat. This extends Ashikhmin et al's results 
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Fig. 1. Serial Concatenation Scheme 



work well with iterative decoding/signal processing [6]. An EXIT chart is usually plotted assuming that 
the a priori LLRs have a Gaussian distribution. But so far the area property has been proved only for the 
erasure case. Therefore designing codes by matching EXIT charts for Gaussian a priori LLRs does not 
have a theoretical justification, although it appears to work well in several cases. 

In this paper, we define a new measure based on the mean squared error (MSE) instead of mutual 
information, and describe an MSE chart similar to an EXIT chart. For this new measure, when the a 
priori information is from an AWGN channel, we theoretically prove an area property that is similar 
in flavor to the area property of EXIT charts in erasure channels. We then use this result to prove that 
matching of the MSE transfer curves of the component decoders is optimal when both the a priori and 
extrinsic LLRs are Gaussian. This result is then extended to prove that EXIT chart matching is also 
optimal. The proof is based on the recent result of Guo, Shamai and Verdu [4] that relates the information 
rate to MMSE and it shows the utility of Guo et a/'s fundamental result. 

We use the area properties derived for the MSE chart to show that for an AWGN channel, the EXIT 
chart of a capacity achieving code is flat. This has recently been proved by Peleg et al in [8]. However, 
the proof in this paper is slightly different from theirs. 

In [1], several different measures used to analyze iterative decoding were studied and it was concluded 
that some measures were robust to different channels. However, in order to compute these measures 
knowledge of the transmitted bits was required and, hence, could not be done at the receiver. We show 
that the measure proposed here is robust and can be computed without knowledge of the transmitted bits. 

The paper is organized as follows. In section HH we present the notation used in this paper. In section Hill 
we outline some existing measures and propose a new measure. In section [IV] we show the area property. 
We prove the optimality of matching for Gaussian LLRs in section |Vj In section IVj we prove that the 
EXIT chart of capacity achieving codes is fiat. We summarize our results in section IVIII 

II. Notation 

We use X to represent a vector and X%,...,X n to denote its elements. We denote a set contain- 
ing elements X h ...,Xj by X{. We use \X\ to denote (^Xf) - 5 . We use 4>{X\Y) to denote the 
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Fig. 2. Exit Chart 



average minimum mean squared error in estimating the elements of X given Y, that is <j>(X\Y) = 

r -> -> -> 2 i 

\Egf X — E-g,f[X\Y] . We drop the subscript in the expectation operator E[] whenever it is 
unambiguous. 

For the AWGN channel Y = ^X + N, with X G {+1,-1} with P(X = 1) = p, and N is 
a Gaussian random variable with zero mean and unit variance, we use I 2 ('j,p) to denote the mutual 
information between X and Y and ^(7, p) to denote the minimum mean squared error in estimating X 
from Y. When p = 0.5 we represent mutual information and MMSE by just 12(7) and 0(7) respectively. 

If Y is the output of an AWGN channel with snr 7 and input X, to highlight that (f)(X\Y) is a function 
of 7 we write it as (p(X\Y, 7). We do not encounter cases where the snr is unknown in this paper. 

We will use A and p to represent the edge perspective degree profile of the variable nodes and the check 
nodes in an LDPC code, where n is the total number of edges, n\i is the number of edges connected to 
degree i bit nodes and npi is the number of edges connected to degree i check nodes. 

III. Measures 

Consider the serial concatenation scheme and the corresponding iterative decoder shown in Fig. [T] Let 
L(xk), L ap (xk) and L ext (xk) be the log likelihood ratio (LLR), a priori LLR, and extrinsic LLR on bit 
Xk. Further, let us assume that the two component decoders produce true a posteriori estimates L(xk) 
based on L ap and any other observation from the channel. It has been observed that the pdf of L{xt) can 
be assumed to be Gaussian with mean mxk and variance 2m, denoted by J\f(mxk,2m). Based on this 
assumption, we plot a curve for each of the decoder blocks. We assume an a priori LLR ~ Af(mxk, 2m) 
and generate extrinsic LLR for the inner decoder. We extract some parameter from these LLRs, F(L), and 
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plot F(L ap ) against F(L ext ). For the outer decoder again we do a similar computation but plot F(L ext ) 
against F(L ap ). This is illustrated in Fig. |2]for F(L) = I(X; L), in which case such chart is called an 
EXIT chart. The path taken by the iterations is also shown in the curve. It is clear from the chart that the 
iterations will converge to the correct codeword if the curves do not cross each other. 

We get different charts depending on the parameter that is extracted. Some of the measures that have 
been considered previously are 

Ml Mutual Information measure used in EXIT charts defined in [5] is given by 

F(L) = I(X;L) (1) 

M2 Fidelity measure was defined in [3] as 

F(L) = 9 = E[x k ■ t&ah(L(x k )/2)] (2) 
M3 In [1] a measure 77 was defined as 

F(L) =t} = E[L 2 (x k )] (3) 

In [1], it was shown that measures Ml and M2 are robust and predict the performance of iterative decoding 
well. Measure M3 was proposed as a measure that could be computed without knowing x k s and, hence, 
could be used at the receiver. However, in [1], it is shown that this measure is not robust. 



A. Proposed Measure 

M4 We propose a new measure </> 

F(L) = 1 - = £[tanh 2 (L(x fc )/2)] (4) 

Any APP decoder computes L ext (x k ) from some channel observations Y and the a prior information 
on bits x± and x k+1 . When the APP decoder is a true APP decoder 



LextiXk) = log — ^ ' ? ^ \ * (5) 

The MMSE estimate of x k given Y and L ap (x^, x^ +1 ) is given by 

x k = P(X k = l\Y,L ap (x k 1 -\xl +1 ))~P(X k = -l\Y,L ap (x k 1 - 1 ,x n k+1 )) (6) 

gLext (#fc) J 

tanh(L ea;f (a; fc )/2) (7) 



I _|_ e L ex t(xk) 1 -4- e L sx t(x k ) 

The MMSE is given by 

E[(x k - x k ) 2 ] = 1 - E[x k .x k ] = 1 — E [(x k ) 2 ] (8) 
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Fig. 3. General additive noise channel 



Therefore we have 

MMSE = = 1-6 (9) 

From the definition © it can be seen that M4 can be computed without knowledge of Since M4 
is equal to M2 when the component decoders are true APP decoders it is robust as well. Let us denote 
the transfer chart obtained using measure M4 as an MSE chart. 

It is easy to see that when both the a priori and extrinsic information are from erasure channels, the 
MSE chart and the EXIT charts become identical. Therefore the area properties derived for the EXIT 
charts in the erasure case also apply to the MSE chart. In the next section we derive some area properties 
for the MSE chart in the Gaussian case. 



IV. Area Property 

In this section we derive some relationships between the rate and the MSE curve of the inner and outer 
code of the serial concatenation scheme shown in Fig. [lj The motivation for the relationships presented 
here is the following result by Guo et al [4] that connects MMSE and mutual information. 

For a Gaussian channel Y = ^JsnrX + iV;where N ~ A/"(0, 1), if X is the MMSE estimate of X given 
Y then 

/(X; Y) = E[(X - X) 2 } (10) 



dsnr 

Using this result when X is binary we get 

^W,p) = ^(i,p) (ii) 

Theorem 1: Consider a system where X is chosen from a code C and transmitted over a Gaussian 
channel with signal to noise ratio 7. Let Y denote the output of the Gaussian channel. Let Z represent 
side information available about X. For this system we have 

f°° - - - - ln4 - - 

/ <f>(X\X e C,Y,Z,j) dj= — H(XXeC,Z) (12) 
Jo n 

where n is the length of the codeword X. 
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Proof: This system is similar to the general additive noise channel model shown in Fig. |3] We have 



I(X; Y, Z\X G C) = I(X; Z\X G C) + I{X; Y\Z,X G C) 



(13) 



Differentiating both sides with respect to 7 and noting that I(X; Z\X G C) is independent of 7 we have 

(14) 



^-I{X- Y, Z\X G C) = -^-I(X; Y\X G C, Z) 

07 0(7 



Given Z the channel between X and Y is Gaussian. By using the relationship derived by Guo et al [4] 
we have 

2 
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I(X;Y,Z\X G C) 
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X|XeC,F,Z, 7 
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0(X|XeC,f, z,j) 



Now integrating both sides with respect to 7 we have 

"°° d 



^7 



I(X;Y,Z\X e C) d 7 



7=00 



I(X;Y,Z\X G C 

G C) - I(X; Z\X G C) 
H(X\Z,X G C) 

0(x|XGC,y,z, 7 ) rf 7 



/(X;F,Z|X G C; 



7=0 
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In 4y 



(15) 



(16) 



Note that in a typical concatenation scheme, X is the input to the inner encoder. However, here we use 
the term inner code to refer to a set of constraints satisfied by X. This difference will me made clear in 
example |2l 

Corollary 1: For any code C of rate R 



(f>{X\X G C,Y,j) dj = R\n4: 



(17) 



where X represents a length n codeword, Y represents the received signal when X is transmitted over an 
AWGN channel with signal to noise ratio 7 and <f)(X\X G C, Y, 7) is the MMSE is estimating X given 
that Y is the received signal when a codeword was transmitted. 

Proof: Follows from Theorem [T] when there is no side information as H(X\X G C) = R. ■ 
To plot the transfer characteristic of a component code, it is assumed that the a priori information is from 
a Gaussian channel. For a true APP decoder, tanh(L(X fc )) = t&nh(L ap (X k ) + L ext (X k )) = E[X k \X G 
C, Y, 7]. For the outer code in a concatenation scheme, the 7 in (fT6l) . corresponds to the SNR of the a 
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priori channel. Hence, if we plot the MMSE at the output, <f>(X\X 6 C, Y, j) — 1 — tanh 2 (L(X fc )), as a 
function of the a priori snr then the area under the curve is equal to the rate of the code times In 4. 

Example 1: In Fig. |4] we plot the MMSE as a function of SNR for different rate 1/2 codes. It can 
be seen that the area under the MMSE curve for the different codes is nearly the same. Numerical 
computations show that the area is nearly In 2. 

In context of iterative decoding, corollary [T] provides a nice relationship between the area under the 
MMSE vs SNR curve and the rate of an outer code. Theorem [T] links the area under the MMSE vs SNR 
curve of an inner code to an information theoretic quantity but its relation to the maximum rate supported 
is not clear. In the following lemma, for a special case, when the outer code is chosen independent of 
the inner code, we derive a relationship between the maximum outer code rate supported and the area 
under the MMSE vs a priori snr curve of the inner decoder. Note however, that this special case is what 
is typically encountered in iterative decoding. 

Example 2: Consider the design of a good LDPC code designed for an AWGN channel with signal to 
noise ratio 7. We can treat this as a concatenated code where X represent the edges and Z the channel 



observations. In this case, the inner code represents the restrictions imposed on X by the irregular repeat 
code (Fig. |5]). The outer code is a single parity check (SPC) code. We are interested in finding a relationship 
between the rate of the SPC outer code and the area under the MMSE chart for the inner irregular repeat 
code. In this case it can be easily seen that the rate of the outer code 1 — X) ^ * s bounded above by 
1 — • (1 — h(l)) = 1 — -H(X\Z). The following Lemma generalizes this result. 

Lemma 1: If an outer code C ou t is chosen independent of the inner code Ci n , then, the maximum 
rate of the outer code that can be used while achieving a vanishing probability of error is given by 
Rout < 1 — -H(X\X G Ci n , Z) where Z represents the channel observation and n represents length of 
X. We will refer to this upper bound as R™ u a £, r - 

Proof: Let m be the length of the outer codewords and let X' represent a length m vector. Consider 
a sequence S of length Nmn. We say S G Ci n if S(ln + 1, • • • , In + n) is a sequence in Ci n for all I. 
Similarly we say S G C ou t if S(lm + 1, • • • , Im + m) is a sequence in C out for all I. We say that Ci n 
and Cout are chosen independently if for a random sequence S the events S G Ci n and S G C ou t are 
independent, i.e., P(S G C in and S G C out ) = P(S G C in )P(S G C out ). 

The number of length Nmn sequences that belong to Ci n is 2 NmH ( x \ XeCin \ Number of length Nmn 
sequences that belong to C ou t is 2 NnH{ - x " [,x ' eC °^ t \ We have 

nNmH{X'\X£Ci n ) 2NnH(X'\X'eC out ) 

P(S G C in and S G C out ) = — — (18) 

and the number of sequences that belong to both C; n and C ou t is 

2NmH{X\X£C in ) 2 NnH ( x ''\X'£C ou t) 
2nmN 



2nmN f f _ 2 NmH ( x \ x &dn)+NnH(X'\X'eC out )-Nmn /jm 



If with some choice of outer code, the decoder is always able to recover S from the channel observations, 
then the total number of sequences S should be less than 2 NmI ( x > z \ x&c ™\ Therefore we have 

NmH(X\X G C in ) + NnH(X'\X' G C out ) - Nmn < NmI(X; Z\X G C in ) (20) 

which implies 

-H(X'\X' G Cut) < 1 - -H(X\X G Ci„, Z) (21) 

m n 

■ 

In the general case (when the outer code is not independent of the inner code), there seems to be no 
such relationship. For example, consider the LDPC code in Example 2 but consider another outer code 
constructed from a good rate R code (R < hil)) by repeating Cj, the jth coded bit, dj times, where 
dj is the degree of the jth bit node. In this case the rate of the outer code is ^ ^-R. Its relationship to 
H(X\Z) is not straightforward. 
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The inner decoder has side information about the coded bits from the channel output apart from the a 
priori information. The transfer characteristics is obtained by increasing the snr of the a priori channel 
from to oo. The outer code and inner code are usually separated by a random interleaver which makes 
the inner code and outer code independent. Therefore from Theorem [T] and Lemma d it follows that for 
an inner decoder, the area under the plot of MMSE at the output against snr of the a priori channel is 
equal to In 4(1 — R), where R is the maximum rate of outer code supported by the inner code. This can 
be easily verified for the following examples. 

Example 3: Consider an uncoded AWGN channel with signal to noise ratio SNR as an inner code. 
Let X be the transmitted bit and let Z be the received signal. Let Y be the output when X is sent over 
another AWGN channel with snr 7. Clearly, MMSE in estimating X from Y and Z is same as MMSE 
in estimating X from the output of an AWGN channel with an snr of 7 + SNR. We have 



00 



<j>(X\Y,Z, 1 )d 1 = / </ ) ( 1 + SNR,p) d 1 
Jo 



00 



<p(j,p) d>y (22) 

SNR 

ln4(H(p)-I 2 (SNR,p)) (23) 



follows from CED- When p = 0.5 we get In 4(1 - I 2 (SNR)). R™£ r = I 2 (SNR) in this case. 
Example 4: Consider an uncoded erasure channel with erasure probability e as an inner code. Let 
equiprobable bits X be the transmitted bits and let Z be the received signal. Let Y be the output when X 
is sent over an AWGN channel with snr 7. We have <j)(X\Y, Z, 7) = (1 — e) • + e<p(X\Y, 7). Therefore 

/ <j>(X\Y,Z n ) d 1 = e / 0( 7 ) d 1 
Jo Jo 

= em 4 From (fTTTl 
Hence R™ter = 1 — e which is exactly the capacity of this channel. 

Example 5: Consider an inner code corresponding to an LDPC code over an AWGN channel. Let X 
represent the edges, Y the a priori messages and let Z represent the channel information at the bit nodes. 
The MMSE for an edge connected to a bit node of degree i is 0(27 + SNR). Let {Aj} and {p^} represent 
the degree profile of the LDPC code in edge perspective. We have 

/•oo r°° 
/ (j){X\X G C, Y, Z, 7) d 7 = VAi / 0(i 7 + SNR) d 7 
Jo i=1 Jo 

= In 4 - i 1 -HS NR)) 



1 

i=l 
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For an LDPC code that works well at SNR, we have 



Ef 

i 

Pi ^ -, 



< HSNR) 



i 

^R outer = l-Y^Pl < 1- V-(l-/ 2 (5iVi?)) 
It is interesting to compare the area property derived here with that derived by Ashikhmin et al in [2]. 



It was shown that the area under the EXIT curve, when both the a priori and extrinsic information can 
be modelled to be from erasure channels, is given by 



Area = ( - ^ H{Xi\X E C) 



' H(X\Z.XeC) 



n 

i=i 



(24) 



Z- =1 H(x t \XeC) 

(l24l was obtained by modifying equations (22), (23) in [2] to suit the notation used in this paper. 

In the special case when H(Xi) = 1 the area becomes 1 — ^H(X\Z, X e C). For an outer code of rate R 
the area is therefore is 1—R. For some specific inner codes Cj n , it was shown that 1 — -H(X\Z, X 6 Cj n ) 
is the maximum rate of the outer code that can be used in iterative decoding to achieve error free 
communication. 

In this paper, we have proved that 1 — -H(X\Z,X E Ci n ) is indeed the maximum rate of outer code 
that can be used for reliable communication when the outer code and inner code are independently chosen. 
This makes the area property derived for EXIT charts more concrete. 

We note that in the case when H(Xi\X £ C) ^ 1, the simple relationship between the area under the 
EXIT chart and rate does not hold. However the relationship between area and rate of the outer code and 
the relationship between area and R^^ r for the inner code continue to hold for the MMSE vs SNR plot. 



A. Area Property for MSE chart 

Let us assume that the bits about which information is exchanged in an iterative decoding scheme 
(usually the coded bits of the outer code) are equiprobable. Further, let the a priori and the extrinsic 
information can be modelled as though the bits were transmitted over an AWGN channel. Let us refer to 
the SNRs of these channels as snr ap and snr ext . We first note that if a true APP decoder is employed, 
MSE is equal to the MMSE. We denote the MMSE corresponding to the a priori, the extrinsic, and the 
output LLR by MMSE ap , MMSE ext and MMSE out respectively. 

We will refer to a plot of MSE ext versus MSE ap as an MSE transfer curve. An MSE chart then has two 
MSE transfer curves, one for the inner decoder and one for the outer decoder. The area properties proved 
so far are for a plot of the MMSE out (not MMSE ext ) versus the snr ap . With the Gaussian assumption, 
the MMSE ou t vs snr ap plot can be generated from the MSE transfer curve using the transformation 
(1 - MMSE ap , 1 - MMSE ext ) -> {<P{<p-\MMSE ap ) + <p-\MMSE ext )), (p~ l (MMSE ap )). The area 
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properties derived thus apply for the MSE transfer curve under this transformation. For convenience, we 
use i a p and i ext to denote 0~ 1 (MMS , £ , ap ) and 4>~ l (MMSE ext ), respectively. We will use subscripts 
inner and outer to refer to quantities corresponding to the inner and outer decoders. 

Lemma 2: If the a a priori and extrinsic information can be represented as information from Gaussian 
channels then for a rate R code we have 4>(ia P + lext) di ext — (1 — R) In 4 

Proof: Consider the transfer curve as a continuous curve from (0, 0) to (1, 1) by connecting any 
discontinuity in <fi(iext) by vertical lines. With every point (x, y) = (1 — <j){i ap ),l — 4>(iext)) on the 
transfer curve associate a variable z = x 2 + y 2 . The reason for introducing this variable is to make it easy 
to handle the possibility discontinuity of the MSE transfer curve. It is easy to see that 7 ap and i ext are 
both continuous and increasing functions of z such that 7 ap and i ext are at z = and oo at z = 2. 

In 4 = / 4>{lap + lext) d(l ap + l ext ) 
Jz=0 

2 p2 

<t>{lap + lext) d'Jap + / 0(7 ap + ^ ext ) djext 
2=0 Jz=0 

OO POO 

0(7ap + lext) dl ap + / <j)(l a p + l ext ) dj^ 

7ap=0 J "(ext=0 



POO 

i?ln4+ / 4>(la P + lext) d>y ext 
Jo 



Lemma 3: For an inner code when the a a priori and extrinsic information can be represented as 
information from Gaussian channels then the maximum supported outer code rate (R^j^.) is given by 

Jo°° 4>(lap + lext) dl ext / (In 4) 

The proof is similar to the proof in the previous lemma. 

Example 6: Consider a repetition code of rate 1/N. In this case when the a priori information is 

from an AWGN channel of snr 7 ap then the extrinsic information can be modelled as information from 

a Gaussian channel of snr (N — l)i ap - We have 

f°° N - 1 f 00 ( 1 \ 

<\>(lap + lext) d^ ext = (N - 1) / (frdap + (N - l)ia P ) dj ap = — / <j>{x) dx = I 1 - — J In 4 

(25) 

This verifies Lemma |2j 

V. Optimality of Matching 

In this section we prove that the MSE curve of the outer code has to be matched to the MSE curve of 
the inner code when the extrinsic information resembles that from an AWGN channel. 

Lemma 4: For two codes C x and C 2 such that 1 - MMSE^ t (i ap ) < 1 - MMSE%{i ap ) V i ap , 
i?x > R 2 with equality only when the two curves overlap. 
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Proof: We have 

MMSE^thap) > MMSE c J t { lap ) 

^lextilap) < lextilap) 
(t>{lap + lex\{lap)) > <P{lap + lextilap)) 
^ R 1 > R 2 

It is easy to see that equality occurs only when the two curves overlap. ■ 
From Corollary [l] and Lemma it follows that a code that is matched exactly to the channel has a rate 

equal to the rate supported by the inner code. Therefore from Lemma |4] it is easy to see that any outer 

code whose flipped MSE curve lies below the inner code and is not matched to the inner code has a rate 

lesser than that supported by the inner code. 

We note that under the Gaussian assumption the MSE curve and EXIT curve are related by a one to 

one function. Therefore since matching is optimal for MSE chart, it is optimal for EXIT charts. 

As a consequence of the results derived so far, under the Gaussian assumptions, the following properties 

hold. 

1) With Gaussian assumption on messages from outer decoder to inner decoder. 

dinner ^lap, inner — 1 — Router (26) 
i i / Pouter ^Ifext, outer 1 Router (27) 

m4 

and for the iterative decoder to converge to the correct codeword <fi} nner < 4>1uter- Here 1 and 2 are 
used to denote the MMSE expressed as a function of the a priori and the extrinsic snr respectively. 

2) With Gaussian assumption on messages from inner decoder to outer decoder, we have 

J dinner &lext,inner = ^outer (28) 

1 r 

~1 I 4*outer d'Jap,outer Router (29) 

and for the iterative decoder to converge to the correct codeword 4> 2 inner > 4>l u ter- 
Depending on the distribution of the exchanged messages, one of the above mentioned properties may 
be used to analyze and design component codes. For example, in an LDPC code the bit to check messages 
closely resembles information from an AWGN channel. In this case we plot MMSE against SNR extrinsic 
for the inner code and against SNR a priori for the outer code. In Fig. |6] we plot these curves for a rate 
0.5 LDPC code that was designed using EXIT charts [6] for an snr of 0.5dB. The degree profile designed 
LDPC code is p 3 = 1, A 2 = 0.254, A 4 = 0.419, and, A 18 = 0.327. The threshold for a bit error rate of 
10~ 4 is .55dB. The threshold predicted using these curves is 0.5 ldB. In Fig. |7]we plot these curve for a 
(3,6) LDPC code. The threshold predicted is 1.05dB and the actual threshold is around l.ldB. 



In 4 
1 
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Rate 0.5 LDPC code designed for AWGN channel 
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Fig. 6. Matching Example 1: LDPC code optimized for AWGN channel 



(3,6) LDPC code 
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Fig. 7. Matching Example 2: (3,6) LDPC code 

VI. Area Property of Capacity Achieving Codes over AWGN channel 

The optimality of matching proved in the previous section assumed that the extrinsic information 
resembles information from an AWGN channel. In this section we prove the optimality of matching for 
the AWGN channel without making any assumption on the extrinsic information. We show that the EXIT 
curve of any capacity achieving code is flat and is matched to the channel. It is also seen that the area 
under the EXIT curve of any rate R capacity achieving code is equal to 1 — R. 

Consider a capacity achieving binary code C of rate R = I 2 (SNR) being transmitted over an AWGN 
channel with signal to noise ratio 7. Since the code decodes perfectly when 7 > SNR the MMSE in 
estimating the transmitted codeword X from the received symbols Y is 0. Therefore from Corollary [T] 



MMSE inner vs SNR extrinsic inner 
MMSE outer vs SNR a priori outer 
100 x (MMSE inner - MMSE outer) 



100 x difference 
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we get 



I rSNR 

R = —J 0(X|XGC,F, 7 )rf7 

i rSNR n 

Jo i=i 

j fSNR n 

— ^0(x 4 |y 4)7 )d7 

, rSNR n 

— ^0(7,P(X, = l)) C / 7 

" /o i=l 

1 - 

= -V^^iV^P^^l)) (31) 

8=1 

< h(SNR) (32) 



-SJV.R ™ 

n In ' 

< ^7 / y>(^ ^ ; (30) 



nln< 



The inequality in d30l is because MMSE error in estimating A from both B and C is always less that 
MMSE error in estimating A from B. It is also easy to prove that 4>(A\B,C) = <p(A\B) only when 
E[A\B, C] = E[A\B]. <ED follows from O- 

Since i? = ^(SNR) the inequalities in d30l and d3TT> have to be equalities. Therefore we have P(Xi = 
1) = 0.5 V i. We also have 



/•SNR pSNR 

/ <P(X l \XeC,Y, 1 )d 1 = / 0(^1^,7)^7 (33) 
Jo Jo 

Since 0(^(^,7) > 0(X;|X G C,Y,j), from 03 it follows that 0(X;|y,7) = 0(X;|X G C,Y,j) for 
almost every 7 G [0, SNR]. 

Now, using the fact that MMSE in both the cases is a decreasing function of 7 and the fact that 
(ft(Xi\Yi, 7) is continuous, it can be shown that 

0(X t |X G C,f, T ) = 0(X,|y, 7 ) 7 < SXR, V i (34) 

It is easy to see that the MMSE estimate E[Xi\X G C, Y, 7] = E[Xi\Y h 7]. Therefore for 7 < SWiZ we 
have 

tanh ( Lap{Xt) + 2 Lext{Xi) ^J = tanh(^^ 

^L ext (X t ) = (35) 

Therefore /(X; L ext ) = when 7 < SXR. 

When 7 > SNR, L ap + L e:rf = +00 when X = 1 is transmitted. Since L ap < 00 we have L ext = +00 
when X = 1. Similarly we have L ext = —00 when X = —1. Therefore /(X; L e:rt ) = 1 when 7 > SNR. 

We note that in [8] a very similar approach has been used to arrive at the same result. The proof 
presented here though is simpler and avoids some of the steps in [8]. 
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VII. Conclusion 

We proposed a new measure based on MSE for analyzing the convergence behavior of iterative decoding 
schemes. This measure is robust and can be computed without the knowledge of the transmitted bits. Under 
Gaussian assumptions, we showed a mapping from the MSE chart such that for any code the area under 
the map is equal to the rate. We used this to prove that curve fitting is optimum in the MSE chart and 
then extended it to the EXIT chart case. For the AWGN channel, without making any assumptions on the 
distribution of extrinsic LLRs, we showed that capacity achieving codes have an EXIT chart that is flat 
and matched to the channel . 
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